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Abstract 

Four  experiments  are  described  in  which  subjects  were 
required  to  choose  among  alternatives  on  the  basis  of  two,  four, 
six,  or  eight  relevant  facts.  Both  decision  quality  and  decision 
time  were  measured.  Presenting  more  than  four  facts  caused  a 
decrease  in  decision  making  efficiency. 
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HUMAN  DATA  PROCESSING  LIMITS  IN  DECISION  MAKING 


J.  R.  HAYES 

INTRODUCTION 

Currently,  the  name  "decision  making"  is  applied  to  a  very 
large  group  of  behaviors  which  range  in  complexity  from  predicting 
which  of  two  lights  is  about  to  light  (  !  ,  Z)  to  establishing  plans  for 
conducting  a  war.  This  report  will  be  concerned  with  decisions 
similar  to  the  decisions  involved  in  choosing  which  one  of  a  number  of 
cars  to  buy  or  which  one  of  several  apartments  lo  reni.  Most 
usually  in  such  decisions,  the  alternatives  will  differ  from  one  another 
in  several  characteristics,  and  these  differences  must  be  taken  into 
account  simultaneously  in  making  the  choice.  For  example,  in 
choosing  among  alternative  apartments,  one  may  considei:  cost,  six.e 
appearance,  convenience  of  location,  quaiiiy  of  neighborhood  and 
jiossibly  a  number  of  other  charactc-risiics..  Tb.c  difficulty  in  making 
such  decisions  arises  in  trading  the  advantage  s  ol  an  aiie rriative  iii  some 
characteristics  against  its  di  s  advarilngrs  in  otlic-r  char  ac  ic  r  isti  cs  . 

Such  decisions  might  be  desc'rihed  as  multi -dimensional  judgments. 

ll  is  ccjrnrnonly  assumed  Ihnl  the  more  rclevani  data  one  takes 
into  account  in  making  a  decision,  the  better  tliat  decision  wil;  he.  It 


is  clear,  however,  that  as  one  takes  more  relevant  characteristics 
into  account  in  comparing  alternatives,  the  opportunities  for  confusion 
increase.  If  confusion  were  to  increase  rapidly  enough  as  the  number 
of  characteristics  increased,  it  is  conceivable  that  decision  makers 
would  perform  better  if  some  of  the  relevant  data  were  eliminated.  The 
experiments  reported  below  concern  the  manner  in  which  decision 
making  performam.t  varies  with  the  number  of  char  ac  ter  i  sli  cs 
presented  in  a  complex  deiision  problem.  Two  aspects  ol  performance 
were  measured  -  decision  “quality"  (to  be  defined  explicitly  below)  and 
decision  time. 

The  Decision  Task 

in  all  ol  the  experiments,  the  subjects,  naval  enlisted  men 
made-  decisions  in  a  simulated  military  problem.  The  task  was  to 
dispatcli  one  ol  several  airplanes  to  investigate  a  reported  submarine 
•sighting.  The  decisions  were  to  be  based  on  information  c.on''atned  in  a 
data  matrix  ol  the  lorm  shown  in  Fig.  1  The  top  row  lists  the  alterna 
live  airplanes  and  the  left  hand  column  lists  several  <  ha  r  ai  le  r  is  ti  c  s  oi 
the  aiif  rnatives.  The  numbei  of  alternative  airplanes  varied  from  two 
to  '■•gM  dc-pendiHg  on  the  experiment,  in  all  expe  :  i  mjcnis ,  problems 
.n'oi'irjg  two  lour,  six  and  eight  c  r.a  r  ai  f  e  r  is  tic  s  were  presented. 

In  all  ol  inc  experiments,  the  chaiail>  nstics  used  were  chosen 
iromi  the  same  list  oi  e.ghl  characteristics,  m  the  order  listed  m 
F.g.  1  lliese  were-  the  qual.ly  ol  the-  pilot,  the  airplane  s  armament 
•here-  miasured  as  the  pe  r  c  i-ntagc-  ol  lull  amunilion  load  on  board),  the 


capability  of  the  airplane  to  make  visual  contact  with  a  surface  object, 
the  speed  of  the  airplane,  the  delay  until  the  airplane  could  take  off,  the 
quality  of  the  airplane's  radar,  the  distance  of  the  airplane's  base  from 
the  target,  and  the  time  available  to  search  for  the  target  once  the 
plane  had  reached  the  target  area.  The  characteristics  listed  above  are 
not  necessarily  the  ones  which  military  experts  would  consider  most 
important,  but  are  sufficiently  relevant  to  provide  the  experimental 
task  with  adequate  face  validity  for  the  subjec  ts  employed. 

For  a  particular  airplane,  each  characteristic  could  take  on  any 
one  of  the  eight  values  listed  in  Table  1.  For  example,  the  speed  of  an 
airplane  could  take  on  values  between  37S  M.P.H.  and  165  M.P.H.  in 
30  M.P.H.  steps  and  the  quality  of  the  pilot  could  lake  on  values  from 
E  (excellent)  to  VP  (very  |)Oor).  The  values  of  each  characteristic 
could  be  ranked  unequivocally  from  one  for  the  "best"  or  most 
desirable  value  to  eight  for  the  "Worst"  or  least  desirable  value.  These 
rankings  are  also  shown  in  Table  1. 

The  eight  cb.aracteristics  do  net  share  a  com.mon  unit  of 
measurement,  nor  does  an  increasing  numerical  value  necessarily 
indicate  an  increase  in  desirability.  For  exam].'le,  higher  numbers  are 
more  desirable  tor  "speed"  but  less  desn’atrle  ioi  "delay'".  These 
projierties  of  the  set  of  r.harat  terislics  make  it  unlikely  that  the  subject 
will  adopt  a  simple  arithmetic  rule  for  arriving  at  decisions  and, 
hence,  will  help  to  insure  that  the  decisions  v/ill  involve  genuinely  multi¬ 
dimensional  judgments, 
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It  is  also  important  in  obtaining  multi-dimensional  judgments 
that  the  range  of  relative  weights  of  the  characteristics  not  be  too 
great.  An  example  will  help  to  make  clear  what  is  meant  by  "relative 
weights".  Suppose  that  for  a  given  subject  an  airplane  with  the  best 
speed  (375  M.P.  H.)  and  the  third  best  delay  (4  min.  )  was  judged  equal 
to  an  airplane  with  the  second  best  speed  (345  M.  P,  H.  )  and  the  best 
delay  (0  min.).  For  this  subject,  a  difference  of  one  rank  unit  of 
speed  balanced  a  difference  of  two  rank  units  of  delay.  Delay,  then, 
had  a  relative  weight  of  50%  with  respect  to  speed.  If  delay  had  had  a 
relative  weight  as  low  as  10%  with  respect  to  speed,  then  the  whole 
range  for  delay,  from  one  rank  unit  to  eight  rank  units,  would  not  be 
sufficient  to  balance  a  single  rank  unit  difference  in  speed.  In  a 
decision  based  on  these  two  characteristics,  the  subject  would  have  to 
take  delay  into  account  only  when  the  alternatives  had  identical  speeds. 
The  ranges  of  values  for  the  various  char ac  teristics  shown  in 
Table  1  were  chosen,  on  the  basis  of  experience  with  preliminary 
tests,  to  yield  a  small  range  of  relative  weights.  For  these  and  other 
rrasons  to  be  discussed  below,  it  is  felt  that  the  range  of  relative 
weights  has  been  kept  suit .Ciently  small  for  the  purposes  of  the  present 
'itudy 

surcs  of  Pc-rlo LmaiTc e_ 

Data  we  re  coller  ted  on  decision  time  and  on  two  measures  of 
deiision  quality;  the-  inlormalion  transmitted  from  the  data  matrix  to 
the  decisions  and  liie  mean  grade  ol  Die  decisions.  The  two  measures 


of  quality  were  intended  to  supplement  one  another. 

The  decision  time  was  the  interval  in  seconds  between  the  moment 
at  which  the  subject  was  told  to  expose  the  data  matrix  and  ihe  moment  at 
which  he  announced  his  decision. 

The  information  transmitted  from  the  data  matrix  to  the  decisions, 

T(M,D),  was  calculated  by  the  technique  of  McGill  (6)  from  the  number 

of  subjects  choosing  each  alternative.  (See  Appendix  B  for  a  discussion 

of  this  technique.)  T(M,D)  was  used  as  a  measure  of  the  quality  of 

decisions  because  its  m.agnitude  reflected  the  degree  to  which  the 

1 

decisions  were  determined  by  the  data  presented. 

While  large  values  of  T(M,D)  would  indicate  that  the  subjects' 

choices  were  strongly  influenced  by  the  data  presented,  a  question  could 

be  raised  as  to  whether  the  choices  were  "good"  ones  or  not.  It  was 

decided,  therefore,  to  su|jplement  the  iniormation  transmission  measure 

*F  ootnote 
1 

Ideally,  one  should  measure  the  information  transmitted  fi  om 
the  data  matrix  to  the  decisions  for  eatdi  subject  rather  than  for  the 
group  of  subjects.  As  Garner  (3)  has  shown,  individual  differences 
may  become  more  marked  as  the  com|)lexity  of  tfU'  iudgmentai 
situation  increases.  The  ])resence  of  such  an  effi'ct  in  the  present  data 
would  cause  a  progressive  under  estimation  of  the  pr  edictability  of  the 
decisions  as  the  number  of  charac. teristics  was  increast'd.  Unfo^iun 
ately,  practicat  limitations  made  if  impossible  to  mcasun-  transmnted 
information  separately  for  eat.h  subject. 


with  a  measure  of  the  "goodness"  oi  the  choices.  The  measure  used 
here  is  called  "the  mean  grade"  of  the  decisions. 

The  mean  grade  was  computed  as  follows:  A  grade  from  zero  to 
100  was  assigned  to  each  alternative  of  each  problem  by  a  procedure  to 
be  described  below.  The  mean  grade  for  a  set  of  decisions  was  the 
average  grade  oi  the  alternatives  chosen  in  those  decisions.  The 
loliowing  procedure  was  used  to  assign  grades  to  alternatives; 

Consider  tne  data  matrix  shown  in  Fig,  1.  Each  value  of  each 
characteristic  in  the  matrix  was  assigned  a  rank  from  one  for  "best" 
through  eight  for  "worst"  as  is  shown  in  Table  1.  Figure  Z  shows  the 
ranks  so  assigned  together  with  the  sums  of  ranks  fer  each  alternative 
airplane.  The  aiternarA  e  with  the  smallest  sum  of  ranks  in  each 
problem  was  designated  "best"  and  given  a  grade  of  100.  The  alterna- 
:ive  with  the  largest  sum  of  ranks  was  designated  "worst"  and  given  a 
grade  of  zero.  Alternatives  beiween  these  two  were  given  grades 
proportional  to  the  difference  In  sum  of  ranks  between  the  given  al  ■ 
ternative  and  the  "worst"  alternative.  The  measure  of  the  quality  of 
a  stt  of  de  c  isions  was  the  mean  grade  of  that  set  of  decisions. 

1:  is  clear  tliat  the  sum  of  ranks  criterion  Ignores  the  inter 
actions  amor.g  the  c  har  acte  r  istic.s  which  the  subjects  might  try  to 
take-  into  account  in  making  their  decisions. 

Ynteina  and  Torgerson  (8)  have  shown  however  that  the  ellc^c! 

01  .gnering  sin  n  inter  aiuions  in  dec  is.ons  oi  the  type  described  here  arc- 
no!  likely  to  be  sc  r.ous. 
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The  sum  of  ranks  criterion  "works"  in  the  sense  that,  on  the 
average,  it  orders  the  alternatives  in  the  same  way  that  the  subjects  do. 
In  the  four  alternative  problems  of  Experiment  I,  for  example,  the 
subjects  chose  the  "best"  alternative  52.  1%  of  the  time,  the  "second 
best,  "  29,  4%  of  the  time,  the  "third  best,  "  12.  2%  of  the  time,  and  the 
"worst,  "  6.  3%  of  the  time.  The  same  tendency  is  reflected  in  the 
behavior  of  individual  subjects.  Every  subject  in  all  of  the  experiments 
reported  below  chose  the  "best"  alternative  more  frequently  than  any 
other. 

Experiment  1 

Subjects:  16  naval  enlisted  men  with  GCT  scores  of  50  or  more. 
Materials  S  The  materials  consisted  of  240  data  matrices  of 
the  general  form  shown  in  Fig.  1  each  printed  on  an  8"  by  10  1/2"  sheet 
of  white  paper.  The  240  matrices  were  made  up  of  12  families  of  ten 
four-alternative  matrices  and  12  families  of  ten  eight-alternative 
matrices . 

The  construction  of  a  family  of  matrices  involved  the  following 

steps : 

1.  The  list  of  eight  characteristics  was  divided  into  four  pairs, 
e.g.  speed-pilot,  distance-delay,  etc 

F  ootnote 

^Details  of  the  methods  used  to  construct  the  m.aterials  for  Experiments 
1,  2,  and  3.  are  given  in  Appendix  A. 
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Z.  Alternatives  were  chosen  to  be  best,  second  best,  third 
best,  and  fourth  best  in  terms  of  the  sum  of  ranks  criterion,  (e.g. 
airplane  5  might  be  best  in  ail  ten  problems,  airplane  3  second  best, 
etc.).  In  the  four  alternative  problems,  there  was  only  one  fourth 
best  alternative,  but  in  the  eight  alternative  problems,  there  were 
live  fourth  best  alternatives  all  with  the  same  sum  of  ranks. 

3.  Values  were  assigned  to  the  characteristics  in  such  a 
way  that  the  sum  of  ranks  lor  each  alternative  over  each  pair  of 
characteristics  was  one  less  than  lor  the  next  best  alternative,  e.g. 
the  sum  of  ranks  over  the  speed-pilot  pair  might  be  8  for  the  best 
alternative,  9  for  the  second  best,  10  for  the  third  best,  and  11  for 
the  one  or  five  fourth  best  alternatives. 

4,  The  seven  standard  matrices  in  a  family  were  con  ■ 
strutted  by  eaterung  one  or  more  of  the  pairs  of  charat.leristics  into 
a  matrix.  The  four  two  characteristic  matrices  each  consisted  of 
one  such  pair.  The  four,  the  six,  and  the  eight,  characteristic 
matrices  consisted  respectively,  of  two,  three,,  and  four,  such  pairs. 
The  four,  six,  and  eight  characteristic  problems,  then,  may  be 
viewc-d  as  a  combination  of  various  of  the  two  charac.teristic  problems. 

3.  The  three  reduced  information  matrices,  consisting  of  one 
lour  one  six,  and  one  tight  charactc'r  isi.c  matrix,  were  the  same  as 
tlie  corresponding  standard  problems  with  one  modification.  One  of 
the  pairs  of  c.har  ac  ter  isli t.s  in  the  standard  matrix  was  modified  so 
ihat  the  sum  of  ranks  over  that  pair  was  the  same  tor  all  alterna  • 
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tives.  Thus,  in  the  reduced  information  problems,  one  of  the  pairs 
was  not  useful  in  locating  the  best  alternative. 

To  summarize,  in  the  standard  problems,  the  advantage  of 
each  alternative  over  the  next  best  alternative  was  proportional  to 
the  number  of  characteristics  involved.  In  the  two,  four,  six  and 
eight,  characteristic  problems,  the  advantage  was,  respectively,  one 
two,  three,  and  four  rank  units.  In  the  reduced  information 
problems,  tlie  advantage  Vi/as  one  rank  unit  less  than  in  the  corre¬ 
sponding  standard  problems.  Table  2.  shows  the  num.ber  of 
standard  problems  and  the  number  of  reduced  information  problems 
with  each  number  of  alternatives  and  each  number  of  character¬ 
istics. 

Procedure 

Before  testing,  each  subject  was  instructed  as  follows; 

"This  is  an  experiment  in  decision  rnaking.  You  are  based 
at  a  shore  station  and  you  receive  ■reports  of  radar  sightings  of 
submarines.  It  is  your  job  to  dispatch  a  single  plane  to  search  the 
area  where  the  sighting  occurred.  To  do  this  you  have  to  decide 
which  one  of  the  available  planes  is  best  lor  the  assignment.  " 

"In  each  problem,  you  will  have  either  four  or  eight  planes 
from  which  you  must  choose.  In  making  your  decision  you  will  have 
to  consider  several  factors  which  describe  the  planes,  " 

At  this  point  the  subject  was  shown  a  data  matrix  listing  the 
eight  characteristics.  Each  was  explained  to  him  in  detail.  The 
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subject  was  then  asked  to  paraphrase  the  explanations.  If  the 
subject's  paraphrasing  was  judged  unsatisfactory,  the  explanation  was 
repeated. 

The  instructions  then  continued  as  follows: 

"The  problems  will  be  presented  to  you  on  these  sheets.  " 

(E  shows  S  a  data  matrix),  "  and  will  be  placed  face  down  in  front  of 
you.  When  I  give  the  signal  turn  the  problem  over  and  start  work. 
When  you  have  finished,  tell  me  which  plane  you  have  chosen 

"Each  day's  test  will  consist  of  20  problems.  In  each 
problem,  you  should  try  to  make  the  best  decision  possible;  accuracy 
IS  the  most  important  thing.  " 

After  instruction,  the  subjects  solved  a  set  of  20  practice 
problems  which  included  all  of  the  problem  types  shown  in  Table  2. 
Each  problem  type  was  represented  with  frequencies  proportional  to 
[hose  shown  in  Table  2.  The  subjects  then  solved  20  problems  a  day 
in  each  of  12  daily  experimental  sessions.  No  session  included  more 
than  one  problem  from  any  problem  family. 

In  no  case  were  the  subjects  told  whether  or  not  their  de¬ 
cisions  wpi'C  "correcc  "  nor  weic^  they  told  h.ow  to  weigh  the  various 
characteristics.  All  subjects  vvt  re  tested  in  individual  sessions. 
RESULTS 

Figure  3  shows  that  decision  time  increased  markedly  as  the 
number  of  char  ac.ier istics  was  increased  both  for  four  and  for  eight 
alternative  problems.  The  signifii.  ance  of  differences  in  decision 
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time  between  problem  types  was  tested  by  two-tailed  sign  test  (7)  with 
a  5%  significance  level.  All  differences  among  the  four  alternative 
standard  problem  types  and  among  the  eight  alternative  standard 
problem  types  were  significant  except  for  the  difference  between  the 
four  and  six  characteristic  problems  with  eight  alternatives.  Decision 
time  was  significantly  greater  for  all  eight  alternative  standard  and 
reduced  information  problem  types  than  for  the  corresponding  four 
alternative  problem  types.  The  reduced  information  problem  types  did 
not  differ  significantly  from  the  standard  problem  types  except  for  the 
eight  alternative  four  characteristic  problems. 

While  decision  time  increased  as  the  number  of  characteristics 
was  increased,  decision  quality  did  not.  Figure  4  shows  that  the 
information  transmitted  from  the  data  matrix  to  the  decisions  did  not 
increase  as  the  number  of  characteristics  was  increased.  In  fact,  a 
fairly  marked  drop  may  be  observed  between  two  and  four  character¬ 
istics  for  the  eight  alternative  problems. 

Similarly,  Figure  5  shows  that  the  mean  grade  of  the  decisions 
did  not  incr'ease  as  the  number  of  characteristics  was  increased. 

The  significance  of  differences  in  mean  grade  was  tested  by  two- 
tailed  sign  tests  (7)  with  a  5%  significance  level.  Mean  grades  for  the 
four  alternative  standard  and  reduced  information  problem  types  were 
all  significantly  greater  than  for  the  corresponding  eight  alternative 
problem  types  exce|it  for  the  standard  six  characteristic  problems. 

None  of  the  differences  among  the  standard  four  alternative  problem 


types  was  significant.  The  only  significant  difference  among  the 
standard  eight  alternative  problem  types  was  the  decrease  in  mean 
grade  from  two  to  four  characteristics. 

DISCUSSION 

The  failure  to  find  any  increase  in  decision  quality  as  the 
number  of  characterisiics  increased  was  unexpected.  It  lead  the 
experimenters  to  test  two  hypotheses  which  might  account  for  the  re  ¬ 
sult.  The  first  was  that  the  measures  of  decision  quality  were 
insensitive  to  real  changes  in  the  quality  of  the  derisions.  This 
hypothesis  was  tested  by  comparing  the  reduced  information  problems 
with  the  standard  problems.  It  was  reasoned  that  the  quality  of  de¬ 
cisions  should  be  higher  for  the  standard  problem  than  for  the  corre¬ 
sponding  reduced  information  problems  since  the  standard  problems 
contained  a  higher  proportion  of  useful  data  than  did  the  reduced 
information  problems.  As  Figures  4  and  5  show,  the  reduced  in 
iormat'on  decisions  were  in  all  cases  lower  in  quality  on  either 
mearure  than  the  corresponding  standard  derisions.  The  difference  in 
mean  grade  between  the  reduced  information  decisions  and  the 
standard  decisions  was  significani  at  the  5%  level  by  two-tailed  sign 
tests  (7).  The*  hypothesis  that  the  results  of  the  experiment  could  be 
attributed  to  insensitivity  of  the  measures  of  quality  was  therefore 
r  ejec  ted. 

The  second  hypothesis  was  that  more  data  could  not  lead  to 
tietler -decisions  in  the  jiar  tic.ular  problems  chosen  lor  study.  One 


might  argue,  for  example,  that  since  the  information  in  each  pair  of 
characteristics  was  in  a  sense  redundant  with  the  information  in  every 
other  pair,  four  pairs  of  characteristics  could  give  the  subject  no  more 
information  than  one  pair.  Therefore,  one  would  expect  no  improve¬ 
ment  in  decision  performance  as  the  number  of  characteristics  was 
incrtas  td. 

The  test  of  this  hypothesis  involved  demonstrating  that  more 
data  could,  in  fact,  be  used  to  produce  better  decisions.  As  rioted 
above,  each  jrair  of  characteristics  'in  each  standard  eight  character¬ 
istic  problem  had  been  presented  separately  as  a  two  characteristic 
problem.  The  four  two  characteristic  decisions  were  used  to  con^ 
struct  a  synthetic  eight  characteristic  decision  by  the  use  of  a  simple 
voting  rule.  That  is,  the  alternative  chosen  in  the  synthetic 
decision  was  the  alternative  most  often  chosen  in  the  four  two 
characteristic  decisions.  When  two  or  more  alternatives  were  tied, 
one  of  them  was  chosen  by  a  random  process. 

Table  3  shows  that  the  mean  grades  over  all  16  subjects  tor 
the  synthetic  decisions  were  higher  than  for  the  standard  two  and  the 
standard  eight  characteristic  decisions.  The  columns  labeled  n  in 
Table  3  show  the  numbers  of  subjects  for  whom  the  mean  grade  for 
the  syntlieiic  decisions  was  greater  tlian  the  mean  grade  for  the 
standard  dr-cisions.  The  differences  between  synthetic  and  standard 
decisions  were  ail  significant  beyond  the  3%  level  by  two-tailed  sign 
tests  (?i.  Since  extra  data,  properly  used,  could  lead  to  better  de- 
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cisions,  the  second  hypothesis  was  also  rejected, 

A  simple  two  factor  theory  is  proposed  to  account  for  the  re¬ 
sults  with  respect  to  decision  quality^  It  is  assumed  that  the  ability 
of  subjects  to  identify  good  alternatives  depends  on  two  factors.  First, 
it  depends  on  the  differences  in  value  among  the  alternatives.  (Value 
is  crudely  represented  in  this  experiment  by  sum  of  ranks.)  Other 
things  being  equal,  the  greater  the  difference  in  value,  the  more 
likely  it  is  that  the  subjects  will  choose  the  best  alternative.  Second 
it  depends  on  the  subject's  sensitivity,  that  is  on  his  ability  to  judge 
which  of  several  alternatives  has  the  greatest  value.  Other  things 
being  equal,  the  subject's  sensitivity  decreases  as  the  number  of 
characteristics  he  takes  into  account  increases. 

Several  of  the  experimental  results  are  consonant  with  the 
theory.  The  differences  between  standard  problems  and  reduced 
information  problems  of  the  same  size  can  be  accounted  for  by  thi 
first  factor.  As  Figures  4  and  5  show,  the  reduced  information 
problems,  characterized  by  small  sums  of  ranks  differences,  all 
y.eld  decisions  of  lower  quality  than  do  the  corresponding  standard 
problems . 

Footnote 

^Tliis  theory  will  be  developed  more  fully  in  a  later  publication.  This 
simple  account  is  given  here  in  an  attempt  to  make  the  presentation 
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One  could  account  for  the  lack  of  differences  among  standard 
problems  by  assuming  that  as  the  number  of  characteristics  was 
increased,  the  advantage  of  the  linearly  increasing  differences  in 
value  was  just  offset  by  a  linearly  decreasing  sensitivity  of  the  subjects„ 
The  assumption  that  sensitivity  decreases  linearly  as  the  number  of 
characteristics  is  increased  is  ad  hoc  at  this  point.  However,  as  will 
be  shown  in  a  later  publication,  this  assumption  is  useful  in  describing 
other  aspects  of  the  data,  e.  g.  the  superiority  of  the  synthetic  de¬ 
cisions,  and  the  tendency  for  the  quality  of  the  reduced  information  de 
cisions  to  increase  as  the  number  of  t  har  ac  te  risti  cs  is  increased. 
Experiment  Z 

Apparently  the  subjects  of  Experiment  1  were  not  able  to  use 
extra  data  efficiently  enough  for  it  to  yield  an  improvement  in  de¬ 
cision  quality.  It  seemed  reasonable  that  this  inefficiency  migh'  be  due 
to  lack  of  training  and  that  training  of  the  subjects  in  decision  making 
might  reduce  it.  Experiment  2  was  performed  to  see  whether  or  no: 
the  decisions  of  subjects  trained  in  decision  making  would  improve  in 
quality  as  the  number  of  characteristics  was  increased. 

Subjects;  The  subjects  were  !(>  naval  enlisted  men  with  G.C.T. 
scores  of  50  or  more. 

Materials:  The  materials  were  the  same  as  those  used  in 
Experiment  1. 

Pr  ocedu  r e 


The  subjects  were  arbitrarily  divided  into  two  groups  ol  eight 


subjects  each,  Group  A  and  Group  B,  and  the  problems  were  divided 
into  two  sets,  Set  1  and  Set  2,  Set  1  contained  six  of  the  12  four 
alternative  problem  families  and  six  of  the  12  eight  alternative 
problem  families.  Set  2  contained  tne  remaining  problem  families. 
Thus,  each  set  contained  12  complete  families  of  ten  problems  each  or 
120  problems.  Group  A  was  trained  on  Set  1  and  tested  on  Set  2 
Group  B  was  trained  on  Set  2  and  tested  on  Set  1. 

The  subjects  solved  20  problems  a  day  in  13  daily  sessions. 
Session  1  was  devoted  to  the  .20  practice  problems  used  in  Experiment 
1.  Sessions  2  through  7  were  devoted  to  the  training  problems  and 
sessions  S  through  13,  to  the  test  problems.  The  initial  instructions 
and  the  procedures  during  test  sessions  were  the  same  as  those  used 
in  Experiment  1.  During  sessions  1  through  7,  however,  the  subject 
was  told  after  each  decision  whether  his  choice  was  correct  or 
incorrect.  The  experimenter  accepted  as  correct  only  the  alterna¬ 
tive  designa’ed  ''best"  by  the  sum  of  tanks  criterion.  Whenever  the 
subje,  t  made  an  inco.'ttct  t.hoice,  he  was  required  to  c.hoose.  again 
until  he  made  the  cor  re  :t  choice. 

RESULTS 

Only  da-a  irom  the  test  sessions  are  reported  here.  The 
s  igri .  1 1  c  ant  c,  ol  diii'ercnces  among  problem  types  was  in  all  cases 
tested  by  two  tailed  sign  test.s  ■  "^l  with  a  S7o  signilicance  le'^/el 

The  results  ol  Experiment  2  .losely  parallel  those  of 
Experiment  1,  Figure  fa  shows:  that  decision  time-  increased  markedly 


as  the  number  of  characteristics  was  increased  both  for  the  four  and 


for  the  eight  alternative  problems.  All  differences  among  four  and 
among  eight  alternative  standard  problem  types  were  significant 
with  two  exceptions.  These  were  the  differences  between  six  and 
eight  characteristic  problems  with  four  alternatives  and  the  difference 
between  four  and  six  characteristic  problems  with  eight  alternatives. 
Decision  time  was  significantly  greater  for  each  eight  alternative, 
problem  type  than  for  the  co  r  res  ponding  lour  alternative  problem  type. 
The  reduced  information  problem  types  did  not  differ  significantly  from 
the  corresponding  standard  problem  types  except  for  the  eighi  alterna¬ 
tive  six  characteristic  problems. 

The  results  concerning  decision  quality  were  also  very  similar 
to  those  found  in  Experiment  1.  Figure  i  shows  that  the  iniormation 
transmitted  from  the  data  matrix  to  the  decisions  did  not  tend  to  in  ¬ 
crease  as  the  number  of  characteristics  was  increased.  Figure  8 
shows  that  while  the  mean  grades  were  all  somewhat  higher  in 
Experiment  Z  than  in  Experiment  1,  they  did  not  tend  to  increase  as 
the  number  of  c.haractc  ristic  s  was  increasi'd,.  None  of  the  dilferences 
in  mean  grade  among  the  standard  (our  or  among  Uie  standard  tight 
alternative  problt'iri  types  w;is  sign. fit  ant.  Mt  an  grades  (or  llie  lout 
ailernative  jrroblem  tyjies  were  significantly  greater  than  for  the 
eight  alternative  problem  types.  The  reduced  iniormation  problems 
yielded  significantly  lower  grades  than  the  standard  problems. 

Synthetii  decisions  wore  constructed  m  the  manner  disc'ussed 


under  Experiment  1,  from  the  two  characteristic  decisions  made  during 
the  test  sessions.  As  Table  4  shows,  the  mean  grades  for  the 
synthetic  decisions  were  higher  than  for  the  two  or  eight  characteristic 
standard  decision.  Only  the  differences  between  synthetic  and  two 
char acteristic  decisions  were  significant,  however.  The  columns 
labeled  n  in  Table  4  show  the  numbers  of  subjects,  out  of  16  for  whom 
the  mean  grade  for  the  synthetic  decisions  was  greater'  than  for  the 
standard  decisions. 

DISCUSSION 

With  the  exception  that  the  mean  grades  for  decisions  in 
Experiment  2  were  somewhat  higher  than  lor  those  .n  Experiment  I. 
the  results  of  the  two  experiments  were  almost  exactly  parallel. 

Training  in  decision  making,  to  the  extent  that  it  was  provided  in 
Experiment  2,  did  not  appear  to  change  the  relations  found  in 
Experiment  1. 

Experiment  3 

Experiment  3  was  performed  to  investigate  the  effects  of 
limiting  decision  time  on  decision  quality. 

Subjects:  The  subjects  were  eight  naval  enlisted  men  with 
G.  C.  T.  scores  of  SO  or  more. 

Materials:  The  materials  were  the  same  as  those  used  in 
Experiment  1  except  that  only  one  of  the  four  two  characteristic 
problems  in  each  family  was  used.  The  one  problem  to  be  used  was 
si  lecttd  by  a  random  process.  Thus,  the  materials  consisted  of  24 
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families  of  seven  problems  each  or  a  total  of  168  problems. 

Procedure 

Rather  than  being  given  unlimited  time  to  solve  the  problems 
as  in  Experiment  1,  the  subjects  were  told,  "There  will  be  a  ten  second 
time  limit  on  each  problem.  When  the  ten  seconds  are  up,  you  must 
give  me  your  decision  immediately  without  further  study  of  the 
problem.  " 

The  subjects  solved  14  problems  a  day  in  13  daily  sessions. 
Session  I  was  devoted  to  practice  problems  and  Session  2  through  13 
to  test  problems. 

In  all  other  respects  the  instructions  and  procedures  were  the 
same  as  in  Experiment  1. 

RESULTS 

Figure  9  shows  that  the  information  transmitted  from  the  data 
matrix  to  the  decisions  tended  to  decrease  in  the  standard  problems 
as  the  number  of  characteristics  was  increased,.  Figure  10  shows 
that  the  mean  grade  of  the  standard  decisions  tended  to  decrease  as 
the  number  of  characteristics  was  increased  in  the  eight  alternative 
case  but  not  in  the  four  alternative  case.  The  decreasing  trend  was 
found  significant  at  the  6%  level  by  non-parametric  trend  test  (5)  in 
the  eight  alternative  case,  but  not  significant  in  the  four  alternative 
case.  The  differences  in  mean  grade  between  the  four  and  eight 
alternative  standard  problems  were  significant  at  the  5%  level  by  two- 
tailed  sign  tests  (7)  for  the  four,  six,  and  eight  characteristic  problems 

1'^ 


hut  not  for  the  two  characteristic  problems. 

DISCUSSION 

With  unlimited  decision  time  as  in  Experiment  1  and  2,  in¬ 
creasing  the  number  of  characteristics  did  not  increase  the  decision 
quality.  With  limited  decision  time  as  in  Experiment  3,  increasing 
the  number  of  characteristics  could  actually  decrease  decision 
quality. 

Experiment  4 

The  problem  material  used  in  the  first  three  experiments  was 
especially  constructed  to  facilitate  analysis.  In  particular,  two 
constraints  were  placed  on  the  standard  problems.  First,  the  differ¬ 
ence  between  each  alternative  and  the  next  best  alternative  was  fixed 
at  one  rank  unit  per  pair  of  characteristics.  Second,  the  problems 
had  uniform  structure;  that  is,  they  were  constructed  by  adding  to  ¬ 
gether  from  one  through  four  two  characteristic  problems  all  of  which 
had  the  same  best  alternative,  the  same  second  best  alternative,  etc. 
Beiause  of  these  constraints  one  may  question  the  generality  of  the 
obtained  results.  Therefore,  they  were  removed  in  Experiment  4. 

Subjects:  The  subjec  ts  were  14  naval  enlisted  men  with  G.  C,  T. 
scores  of  SO  or  more. 

Materials:  The  materials  consisted  of  300  two  alternative 
problems  eciually  divided  among  the  two,  four,  six,  and  eight, 
characteristic  types.  That  is,  there  were  75  problems  of  each  type. 
The  "^5  eight  characteristic  problems  were  selected  from  a  large  set 
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of  problems  which  had  been  constructed  by  assigning  values  at  random 
to  each  of  the  eight  characteristics.  The  selection  was  made 
randomly  with  the  restriction  that  the  rank  unit  difference  between  the 
alternatives  be  either  one,  two,  three,  five,  or  eight,  and  that  each  of 
these  differences  be  represented  in  15  problems. 

Two  characteristics  were  subtracted  at  random  from  each 
eight  characteristic  problems  to  generate  the  75  six  characteristic 
problems.  In  the  same  way,  the  six  chara.cte ristic  problems  were 
used  to  generate  the  four  characteristic  problems  and  these  in  turn 
were  used  to  generate  the  two  characteristic  problems.  No  effort 
was  made  to  control  the  rank  unit  differences  between  the  adterna” 
tives  in  the  two,  four,  and  six  characteristic  problems.  The  columns 
labeled  h  in  Table  5  show  the  numbers  of  problems  with  each  rank 
unit  difference  for  each  number  of  characteristics. 

Procedure 

The  instructions  were  the  same  as  in  Experiment  1  with  the. 
exception  that  the  subjects  Vv'cre  told  tliat  there  would  be  rwo  aiterna- 
tive  airplanes  in  each  problem  rather  than  four  or  eight.  The 
subjects  solved  ZO  problems  a  day  in  each  of  16  daily  sessions. 

Session  1  was  devoted  to  practice  problems  and  sessions  2  through 
16  to  test  problems. 

RESULTS 

Figure  H  slwwn  decision  time  as  a  function  of  the  rank  unit 
difference  bctv'een  alternatives.  Number  of  characteristics  is  the 
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parameter.  Mean  decision  times  were  computed  only  for  points 
representing  three  or  more  problems.  As  in  Experiments  1  and  2, 
decision  time  increased  markedly  as  the  number  of  characteristics 
'  was  increased.  Decision  time  appeared  to  decrease  somewhat  as  the 

rank  unit  difference  between  the  alternatives  was  increased, 
especially  in  the  four  and  six  characteristic  problems.  The  effect 
was  much  less  marked,  however,  than  for  changes  in  the  number  of 

I 

characteristics.  This  result  confirms  the  finding  of  Experiments 
I  1  and  2  that  decision  time  was  about  the  same  for  standard  and  re- 

I 

I  duced  information  problems  with  the  same  number  oi  character- 

I 

IS  tics. 

I  The  columns  labeled  P  in  Table  5  give  the  observed 

1 

I  probabilities  of  choosing  the  "better"  of  the  two  alternatives. 

I 

Probabilities  were  computed  only  for  groups  of  three  or  more  problems. 
In  Fig.  12,  these  probabilities  were  plotted  on  a  cumulative  normal 
scale  (4)  against  the  rank  unit  difference  ))er  char  acteristic ..  It  is 
apparent  that  the  points  lie  approximately  on  a  single  straight  line. 

Thus,  when  the  rank  unit  difference  per  characteristic  was  held  constant 
the  probability  of  choosing  the  "bc'trr"  alternative  did  not  change  as  the 
numbei  of  characteristics  was  increased.  Since  the  mean  grade  In  a 
two  alternative  decision  is  just  the  prohabilitv  of  choosing  the  "belter" 
alternative  multiplied  by  100,  it  is  clear  that  this  result  confirms  and 
extends  the  results  for  the  standard  problems  found  in  Ex|ieriments 

I  and  2  Previously,  the  result  had  been  obtained  lor  problems  witli 

>  ) 


uniform  structure  in  which  the  rank  unit  difference  per  characteristic 
had  been  fixed  at  0.  5.  In  the  present  study,  the  result  was  obtained 
for  problems  with  random  structure  and  a  wide  range  of  rank  unit 
differences  per  characteristic. 
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APPENDIX  A 


Method  used  in  constructing  the  240  data  matrices. 

Step  1.  Construction  of  12  standard  8-alternative-8-characteristic. 
problems  and  12  standard  4-alternative-8-characteristic  problems 

a.  Best,  second  best,  and  third  best  alternatives  were  chosen 
for  each  problem  by  a  random  process.  The  remaining  5 
alternatives  in  the  eight  alternative  problems  and  the  re 
malning  one  alternative  in  the  four  alternative  problems  were 
designated  "worst.  " 

b.  The  eight  characteristics  were  divided  inio  four  pairs  oi 
two  characteristics  each.  For  six  of  the  four-alternative 
problems  and  six  of  the  eight-alternative  problems  the  pair  • 
ings  were  speed-pilot,  distance- delay  armament-search,  and 
c ontact- 1  adar  For  the  remaining  problems,  the  pair¬ 
ings  were  speed-distance,  pilot-delay,  contact- s earc h,  and 
armament-radar. 

c.  Within  each  problem  a  quality  level  was  chosen  tor  each 
pair  of  characteristics.  This  was  done  by  choosing  a  random 
number  between  7  and  13  for  each  of  the  ;our  pairs  oi 

char  acter  is  tics  within  each  problem.  The  number  c.hosen 
S,  represented  the  sum  of  ranks  over  the  two  character  ■ 
istics  of  th(-  pair  to  be  assigned  for  the  worst  alter  natives. 

The  best,  second  best,  and  third  best  alternativ  es  wi-re 
assigned  sums  ol  ranks  of  S  -  3  S  -Z.  and  S  1  i  espectiv  ely . 
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The  quality  level  was  varied  in  this  way  from  problem  to 
problem  and  within  each  problem  so  that  the  subjects  could  not 
learn  to  identify  the  best  alternative  by  an  absolute  judgment 
of  quality.  With  the  procedure  described  abo'''e,  the  worst 
alternative  in  one  problem  could  be  oi  better  quality  tt.an  che 
best  alternative  in  the  next  |rroblem. 

Table  6  illustrates  the  procedures  up  to  this  point  i’or 
the  data  matrix  shown  in  Fie.  1 

o 

d.  Each  characteristic  was  then  assigned  a  rar.K  for  each 

alternative.  The  ranks  were  assigned  at  rai.dom  with  the 
restriction  that  for  each  pair  oi  (  haracicr  istics  '.he  rar.Ks  ado 
to  the  sum  de.signated  in  c  above  Tlio  sums  oi  :  anKs  used  in 
constructing  the  data  maiiix  m  Fig  1  are  given  in  Table  c 
The  first  cell  of  Fig.  6  shows  that  the  sum  oi  ranks  assigned 
for  speed  and  pitot  was  Id  lor  the  nrst  alternative 
Therefore  a  random  choice  was  made  among  the  pairs  ol 
ranks  -1-^!,  b-o,  7  S,  and  '•’>  A  Table  ''  show.s 

tliat  che  choice  was  (>  and  (> 

e.  The-  order  in  wliicli  the  characti  nstics  were  lusted  in  the 
matrix  was  rhc'n  r andc.un r/.ed .  Thi  procedure-  lo  tins  po.ni 

IS  illustrated  in  Fig  d 

f.  The  assigned  reinks  wore  ti.en  translated  .nto  ateygory 
values  by  the  use  ol  Table'  I  F.guic  I  lilustratc.s  a  eomplfted 
data  matr  ix. 
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Step  Z.  The  construction  of  two  alternative,  four  alternative,  and 
six  alternative,  standard  problems. 


a,  In  d  of  Step  1  above,  a  matrix  such  as  that  shown  in  Table 
8  was  generated  lor  each  eight  characteristic  problem.  The 
procedures  of  Step  2  start  with  these  matrices.  Each  eight 
char  acre  r istic  matrix  was  used  to  generate  four  two 
c  har  ac  leristic  matrices  simply  by  placing  each  pair  oi 
c  hararierisiu  s  in  a  separate  matrix.  Four  and  six  character¬ 
istic  matrices  were  generated  by  selecting  respectively  two 
and  three  of  the  lour  pairs  ol  characteristics-  at  random. 

Thus,  each  eight  characteristic  matrix  was  used  to  generate 
SIX  new  matr  u  es 

b  The  order  in  which  the  c  har  acte  r  is  tic  s  were  listed  was 
randomized  and  the  assigned  ranks  were  translated  into 
category  values  by  the  use  ol  Table  1. 

c  In  plac  ing  the  data  in  the  matrices,  no  gaps  were  left. 

For  example,  .n  two  c  harac  ic  rislic  problems,  the  two 

(.  ha:  ai  terislic  s  were  placed  in  the  top  two  rows  of  the  matrix 

the-  remainder  ol  the  matrix  remaining  blank. 

Step  3..  riic  construction  ol  lour,  six,  and  e.ght  characteristic 
rc'dncc-d  .nloiiiialion  problems. 

a.  Each  lour  six  and  eight  c  harac  lerislic  standard  problem 
was  used  to  generate  a  i  o r  r es |)ondlng  reduced  information 
problem.  One-  pair  ol  characteristics  m  each  standard 
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problem  was  sleeted  at  random  for  modification. 

b.  The  modification  consisted  in  making  the  best,  second 
best,  and  third  best,  alternatives  within  the  selected  pair 
equivalent  to  the  worst  alternatives.  This  was  done  by 
making  the  sums  of  ranks  selected  in  c  of  Step  1  for  the 
three  favored  alternatives  all  equal  to  the  sum  of  ranks 
for  the  worst  alternatives.  Thus,  if  the  distance-delay 
pair  of  Table  6  had  been  selected  for  modification, 
alternatives  2,  5,  and  R  would  then  be  assigned  a  sum  of 
ranks  of  1 1 . 

c.  New  values  of  the  selected  pair  of  characteristics  were 
assigned  to  the  modified  alternatives  by  applying  procedures 
d  and  f  of  Step  1. 

d.  The  order  in  which  the  characteristics  were  listed  in 
the  modified  matrix  was  then  randomi7,ed  again,  so  that  the 
order  of  listing  in  the  reduced  information  problem  was 
different  from  that  in  the  corresponding  standard  problem. 


APPENDIX  B 


The  information  transmitted  from  the  data  matrix  to  the 
decisions,  T  (M,D),  was  measured  as  follows:  First,  the  number 
of  subjects  choosing  each  alternative  was  tabulated  for  each  problem 
as  shown  in  Table  8  Since  each  subject  made  only  one  decision  in 
each  ptoblem,  F.  ,  ~  F.  ~  .  F.  ~  s,  where  s  is  the  number  of 

subjects . 

Second,  T  (M,D)  was  computed  from  the  formula  T  (M,D)  = 


where 


and 


H  (M) 


H  (D) 


H  (M)  +  H  (D)  -  H  (M,D) 

F-t.  «  F^, 

TT 


Jc'.  / 


N 


H  (M,D) 


t,  f , 


For  a  fuller  discussion  of  this  technifjue  see  McGill  (6). 

T  (M,D)  is  used  as  measure  of  decision  quality  because  it 
reflects  the  degree  to  which  the  decisions  are  determined  by  the  data 
matrix.  If  the  choices  were  made  without  reference  to  the  data  matrix, 
then  T  (M,D)  -  0,  Increasing  influence  of  the  data  matrix  on  the 
decisions  will  yield  increasing  values  of  T  (M,D)  up  to  a  maximum 
value  of  log  w'lierc  k  is  the  number  of  alternatives.  Good  decision 
processes,,  of  course,  should  be  influenced  strongly  by  the  data,  and, 
hence  should  yield  large  values  of  T  {M,D). 
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Figure  2 

Ranks  and  Sums  of  Ranks  for  the  Matrix 
Shown  in  Figure  1 
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TRANSMITTED  INFORMAI 


Figure  4 

Information  Transmissions  in  Experiment  1 
- - 1 


2  4  6  8 


CHARACTERISTICS 


32 


CHARACTERISTICS 


MtAN  DECISION  TIME  IN  SECONDS 
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TRANSMITTED  INFORMATION  IN  BITS 


Figure  7 

InformaHon  Transmissions  In  Experiment  2 
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Figure  8 

Mean  Grade  of  Decisions  In  Experiment  2 
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4  Alternatives  8  Alternatives 


Type 

Standard 

Reduced 

Information 

Standard 

Reduced 

Information 

Number 

2 

48 

-- 

48 

-- 

of 

4 

12 

12 

12 

12 

Characteristics 

6 

12 

12 

12 

12 

8 

12 

12 

12 

12 

. 

84 

36 

84 

36 
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TABLE  3 


Standard  Two  Standard  Eight 
[Characteristics  Characteristics  Synthetic 


— 

Mean 

Grade 

n 

Mean 

Grade 

n 

Mean  Grade 

4  Alternatives 

76.  6 

15 

77.  1 

13 

82.  5 

8  Alternatives 

70.  4 

16 

6S.  8 

14 

80.  3 
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TABLE  4 


Standard  Two  Standard  Eight 

Characteristics..  Characteristic 

.  _ 

s  Synthetic 

Mean 

Grade 

n 

Mean 

Grade 

n 

Mean  Grade 

4  Alternatives 

82,2 

16 

83.  6 

10 

89.  5 

8  Alternatives 

76.  4 

15 

79.2 

10 

8'6.  5 
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DIFFERENCE  IN  RANK  UNITS 


Table  5 

Probabilities  of  Choosing  the  "Better"  Alternative 
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